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ENHANCING TECHNOLOGY TRANSFER OF COMPUTER HARDWARE AND SOFTWARE 
ARCHITECTURES USING HUMAN FACTORS IN INITIAL DESIGN 

BACKGROUND 

Technology advances, particularly in the field of automation, 
have far outstripped the capacity of operational users to recognize and 
assimilate all the possible benefits. The research community has 
produced a large variety of capabilities which may be categorized as 
"decision aids" and/or "operational aids" and has concurrently 
experienced considerable difficulty in effecting technology transfer, 
i.e. acceptance of the research by the user or operational community. 

In October, 1977, the Naval Postgraduate School (NPS) in 
Monterey, California, established a new curriculum in Command, Control 
and Communications (C3). This curriculum, as described in the Command, 
Control and Communications (C3) Program section of the Naval 
Postgraduate School Catalog (1978), includes within its objectives, a 
major goal "... to provide the students with enhanced capabilities to 
operate effectively in such diverse areas as military decision making, 
current and future C3 systems design, crisis management, ... ". Two 
courses within the C3 curriculum, OS 3671: Man-machine Interaction, and 
CS3750: C3 Exercise Laboratory, are specifically designed to introduce 
research that is directed toward future C3 systems. As such, they 
represent unique vehicles for studying the relevance of research 
products in that the C3 student body, composed of professional military 
officers, experienced in operational requirements, is exposed to the 
various research products and then provided the class time and equipment 
to use, experiment with, and evaluate each research product. 

The C3 curriculum has received support from the Defense 
Advanced Research Projects Agency (DARPA), a Department of Defense (DoD) 
agency specifically tasked to conduct exploratory research in advanced 
technology. Much of the research in command, control and communications 
is a product of DARPA's Information Processing Technology Office. These 
research products fall into a variety of areas such as graphics 
languages and protocols, distributed data base concepts, production rule 
systems, and natural language query systems as examples. However, it is 
often difficult to evaluate research products like these from an 



operator's perspective and when this is done, the product is often so 
far developed that it is infeasible to change it ... once again, a 
reminder of the need to consider human factors in initial design stages 
of any computer hardware or software development. 

With the establishment of the C3 curriculum and the close 
association with DARPA, a unique situation had developed in which 
military officers in the C3 curriculum would be able to use and evaluate 
emerging technologies as they develop and provide operational user 
feedback to the research community. To enable this to occur, each 
officer in the C3 curriculum has been provided with a portable, hardcopy 
producing, computer terminal. A local telephone call allows the officer 
to connect into the ARPANET, a distributed network of over 100 different 
large scale computers. The geographically separated host computers are 
linked together by a communications net to form an operational, resource 
sharing network of a wide variety of DARPA sponsored research centers 
and other DoD activities in the continental United States, Hawaii, 
Norway and England. 

Originally implemented in 1969 under a research and development 
program sponsored by DARPA (then ARPA) to advance state-of-the art 
computer internetting, the NET'S use has experienced growth in many 
unplanned directions. The ARPANET'S communication system uses the 
technique of "packet switching" where each message is separated into 
small packets and each packet is then switched to its destination via 
the fastest communications path available at any instant. Upon arrival 
at the destination, the packets are reassembled in the proper order. 

Since DARPA 's Command, Control and Communications research 
products reside on host computers around the ARPANET, the officers in 
the NPS C3 curriculum can therefore access these research technologies 
at any time. For example, a student officer can work with a given 
research technology at a computer in California and within seconds be 
working on a real time, interactive basis with a different research 
technology which may reside on a host computer in Pennsylvania. 
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OBJECTIVE 



The authors had both encountered a variety of computer systems 
before but one outstanding feature of the ARPANET and most of the 
Command, Control and Communication technologies on it is that they have 
been designed with much consideration for the operational user. 
(Finally human factors consideration in the design stage!) This was one 
of the primary reasons for deciding to introduce the new C3 class to the 
ARPANET and its C3 technologies. 

Many students find most computer systems too detailed and 
exacting, e.g., if a dollar sign, slash, asterisk, etc., is not entered 
in exactly the proper format, the job will fail and the students become 
very frustrated and "turned off" because the system is not friendly. 
This feeling is further accentuated in military officers, especially 
those in the C3 curriculum, who represent the Army, Navy, Air Force and 
Marines, and who may be operational users of computer systems but who 
have no desire to worry about system details like dollar signs. They 
want to be able to get their operational answers from the computer 
system as quickly as possible, and then get back to their operational 
duties. 

It was therefore the objective of this study to introduce a 
group of relatively naive computer users to a computer network system 
(the ARPANET and its technologies) which was very user oriented in 
design. The question to be addressed was would an educational exposure 
to such a system change the students' perceptions and attitudes toward 
computer systems, and if so, how? 



METHOD 



Subjects 



The 


subjects 


were 


members 


of 


the 


first 


class in the C3 


Curriculum 


at NPS. 


All 


branches 


of 


the 


U. S. 


Armed Services were 



represented with ranks ranging from a Navy Ensign to a Marine Corps Lt. 
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Colonel. Prior to this study, each student had just completed an 
introductory computer programming (FORTRAN) course and an introductory 
computer graphics course, typical of those found in many universities. 
In those courses they had done the usual types of tasks such as writing 
specific programs, setting bits to adjust vectors on cathode ray tubes 
(CRTs), and doing term projects. In general, they had a very common 
computer oriented background. Most of them thought what they had 
learned so far was "nice to know" but they really had no desire to work 
with automated systems and learn more of the nitty-gritty details. 

Procedure 

The subjects were all given a questionnaire prior to the 
beginning of the experiment and prior to receiving their personal, 
portable, computer terminals or any information about events or topics 
coming in the future related to this experiment. 

Subjects received no monetary rewards for participation in the 
following activities. 

Standard background information about operational experience and 
previous computer experience, ages, rank, etc., were obtained from the 
questionnaires. In addition, 30 questions were asked beginnning with 
the phrase "To What Extent ... " with responses A through E ranging from 

A. None, or very little 
to 

E. A great deal 

These 30 questions tried to determine the subjects' perception 
of the value or utility of automatic data processing and automated 
decision/operational aids. The questions covered a broad area and were 
similar to the following: 

"To what extent do you feel you could have effectively 
used a computer or computer network in your past 
jobs?" 
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Questionnaire responses were received from all subjects and not 
examined until the end of the experiment. 

The subjects were each then given a portable, hard-copy, 
computer terminal to take home in addition to a small introductory 
pamphlet (Holg, 1977) explaining the ARPANET in general, and one of its 
message handling systems. They were told a class would begin in four 
days in which they would make use of the terminals on the ARPANET. 

The actual experiment then began in which the students were 
exposed to the ARPANET and its command and control research technologies 
for a period of eleven weeks in a course called "Man-machine 
Interaction", which met three hours per week. During class, video 
monitors were installed around the room so the students could see the 
professor actually working on a real-time, interactive basis with the 
ARPANET and various computers around the country. Students were given 
specific assignments which they would then work on at home on their own 
computer terminals. Since each user on the ARPANET has an individual 
electronic "mail-box", students transmitted their finished assignments 
to the professor's mail-box at anytime they wished. In addition, the 
professor also worked on the NET three nights a week from home. Because 
the ARPANET provides a teleconferencing feature between users, when a 
student had a question, he/she could simply link with the professor and 
receive immediate feedback on any question. Furthermore, if the 
professor noted several students with the same questions or problems, he 
would have them all link together and then go to the particular host 
computer around the country where the students were having questions. 
He would run through a short demonstration to illustrate t e answers to 
their questions, and during this time, each of the students* terminals 
would be printing out an exact copy of what the professor was doing. 

Some of the specific research products and some of their human 
engineering features which the students were exposed to are listed 
below: 
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A small 



1) The ARPANET and some of its general features, 

pamphlet showing how to access the ARPANET and use one of its mail 
handling systems was given to each student. The pamphlet showed the user 
how to access the network and then how to go to his own host computer on 
the network. The pamphlet explained in simple terms what was happening 
and the user entered given commands through his terminal and saw 

comparable results coming back from the system as those shown in the 

pamphlet. The ARPANET protocol makes extensive use of an escape 
character wherein the user can start to enter (log in) to the system and 
by typing one or more escape characters, the system knows who is logging 
in and therefore automatically fills in account data, etc., for the 

user. 

(NOTE: The human engineering features and user designed interfaces of 
the ARPANET were quickly re-emphasized to the authors after they had 
handed out the terminals and the small introductory pamphlet to the 

students several days before class began. Within thirty minutes after 
they left one of the authors' offices with their terminals, the authors 
were also logged in to the system and noticed students starting to lost 
in and use the mail handling system. That was impressive considering 
that the host computer they were using was some 400 miles away and they 
had received no instruction.) 

2) A message handling system (MSG) - which basically transmits 
pages of information from one user to other users' electronic mailboxes 
in the form of messages or letters. Particular features which make this 
system especially useful are its flexibility and its automatic 
recognition mode. If one wants to send a message, one simply types an 
"s" for the command SEND and the system automatically fills out the word 
and asks for the addressee and the host computer name on the network. In 
addition, it asks for addresses of people you may wish to send "carbon 
copies". The pressing of one key, after the message is composed, starts 
the mailing system which sends the message to all addressees specified 
and the system shows the status as OK when each user has been sent the 
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message. If a communication link is down, the message will be listed as 
queued, so the user knows its status. Other features of the mail system 
require only one key to be typed to invoke them such as "d" for deleting 
a message, "f" for forwarding a message to someone from someone else, 
"a" for answering a message, "u" for undeleting a message which the user 
may have marked for deletion by mistake, etc. All messages in the user's 
mailbox are numbered, so to answer message 23, the user simply types 
a23 

and the system fills in the addressee because it knows whose message the 
user is answering. 

3) A text preparation and editing system (XED) - which is used 

for preparing pages of text for longer letters, reports, etc. The 

system makes use of automatic recognition by letting the user type "a" 
for example, and the system fills out the word "append" meaning the user 
wants to add more text, etc. By typing one key, the user can also 
format his text into neat pages with right and left margins aligned. If 
the user wishes to save the text, he simply types "w" and the system 
fills in "write to output file" where the user fills in the file name. 

4) Multi-host computer networking (TELNET) - a system for going 

from one's host computer to another on the network, obtaining some 
information, and subsequently returning to one's own host. For example, 
one may be working at his own host computer and decide he needs 

information from another computer, thousands of miles away. Instead of 
having to terminate operations at one's own host, go log in at the other 
to get the desired information and log out and then log back into one's 

own host again, this system allows one to type two words and a 

connection is made to the remote host from the user's host computer. 

The user can get the desired information and return to his own host in 

exactly the same place he left it. The speed and procedure with which 

one can interconnect several computers is incredibly easy to the naive 

computer user. 
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5) File transfer between host computers (FTP)- a system for 
allowing one user to retrieve or send files of information from or to 
another user. For example, one user wants a 50 page computer program 
which someone else has at a host computer thousands of miles away. By 
typing three words, a connection is made from one host to the other and 
the 50 page computer program at location X can then be transferred to 
location Y in seconds. The user at location Y can now execute that 
program immediately on his host computer. 

6) Search and rescue demonstration (SAR) - a program which 
allows users to specify a disaster site and subsequently initiate search 
and rescue operations. With mostly YES and NO responses, plus some 
numbers entered to specify distances, the user actually sees automatic 
connections being made around the computer network to notify specific 
people of the simulated disaster, to obtain weather data in the vicinity 
of the disaster from other computers, and to determine which ships might 
be in the area to help in this search and rescue on the high seas. 

7) Inquiry programs - to determine what ships are at sea, where 
they are, what they have onboard, etc. Two inquiry systems were 
actually used. One was a "natural language” query system which 
approaches the ultimate in man-machine interface by some standards. It 
allowed the user to ask questions in his own words and the user could 
then see the program trying to parse or understand his question. If 
unsuccessful, the program replied with reasons why it didn’t understand. 
If successful, the user would see the program connect to other 
distributed data bases around the country to obtain the answers. In 
many cases, this system also corrected user typing errors in input. 
From a man-machine interface viewpoint, this system is very flexible in 
that one user can ask n Where is the John F. Kennedy?” and another user 
can ask "What is the position of the JFK?” and both users will get the 
same answer. 

The students also used another query system which was somewhat 
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in contrast to the natural language system and built on a different 
philosophy of the best man-machine interface. In this system, a tree 
structure is used in which a user learns certain key words and tree 
branches. The system then uses automatic recognition to help fill in 
the user’s request. For example, a user might wish to say, 

"Show Russian submarines within 350 nautical miles of Honolulu". The 
user would actually type the following letters: S R SU W 350 N Honolulu 
and the system would fill in the words to make the sentence appear on 
the terminal as written above in the quotes. With a few hours training, 
the user can basically use this system in a sort of short hand way by 
just entering the first letter of words and the system automatically 
types out the rest of the words plus some others, i.e. if the user types 
N, the system prints 

"Nautical miles of...". 

The above descriptions are brief but intended to show some of 
the types of research products the subjects were exposed to, and to give 
an idea of some of the man-machine interface characteristics which had 
been designed into the original product. 

It is important to note here that the authors, in using the 
above systems and research products, had attempted to take these systems 
to the user, who as a naive computer expert, could exercise and run them 
successfully with few problems. The systems introduced were realistic 
products which the subjects could relate with, versus typical 
introductions to computers in which a certain example is used and the 
user is required to imagine and translate this into how it could be used 
in his/her daily tasks. In addition, these were systems which had been 
designed initially with the user in mind, versus many systems designed 
to do a task, and then the user must find out all the small details to 
make them run. For example, in almost every system described above, 
typing HELP or a question mark will provide general assistance on the 
system or answer a user’s specific questions. 
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RESULTS 



The same questionnaire that was administered before the 
experiment was given at the conclusion of the course with all subjects 
responding. The purpose was to determine if the exposure to these user 
designed systems had positively influenced the subjects' perceptions and 
if so, in what direction. 

The responses to the questions had been on a scale from A (None 
or very little) to E (A great deal) and were therefore considered an 
ordinal measurement. Since the magnitude of the change in responses was 
not considered meaningful (a change in response from C to E was not 
necessarily equivalent to a change from B to D), the authors evaluated 
the subjects’ responses to each question with the Sign Test (Siegel, 
1956). This test basically showed if a significant change had occurred 
in the group's opinion during the experiment. A significance level of 
.10 was used since the authors were looking for general trends in the 
subject group's opinion. 

Subjects showed a significant, positive change in response to 
the question "To what extent are you interested in personally working 
with computer systems?". After the experiment, they were more 
interested in personally working with such systems. The reader will 
remember that before the experiment started, the subjects had no desire 
to really get in and work with such systems, even though they had 
previously received a course in FORTRAN and a course in elementary 
graphics. 

The subjects also showed a significant positive increase in 
their perceptions of how they could have used computers in their past 
jobs and in future jobs. After the experiment they felt more strongly 
that they could have effectively used a computer in past jobs, and 
further they now felt more strongly that it would be practical to expect 
future military commanders to use personal computer terminals. On a 
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related question, they also had a much more significant positive 
attitude after the experiment that they would prefer to use interactive 
terminals themselves rather than having a subordinate perform this 
function . 

When asked "To what extent do you view the utility or value of 
computer systems?", the subjects showed no change of opinion. This was 
rather perplexing until the group of subjects was interviewed a few 
weeks later and they replied "We always thought computer systems were 
valuable and we still do... but now we want to be involved with them 
whereas before we really didn't." 

Subjects also showed a significant increased perception of the 
benefit of computerized decision aiding systems. This was probably 
because they had been able to experience such systems and gain 
confidence in the ease with which the systems could be used. They also 
probably gained confidence in the reliability of the systems. 

There was only one question in which the subjects' perceptions 
significantly decreased on the rating scale. The subjects now felt it 
was less likely for them to possibly misunderstand someone else's 
written message. Although the experiment did not include any 
information on how to compose or understand a written message, the 
subjects may have thought they understood messages better because on the 
few times when they did have a question, they could often link up with 
the message sender, wherever that person was on the ARPANET, and get an 
immediate clarification of any question they had about the sender's 
message. 



After the experiment, the subjects also showed a significant 
increase in their responses to the question "To what extent do you find 
it easy to quickly send the same information to many different people". 
Indicating it was now easier was probably due to the fact that in 
previous experiences, they used the typical business approach of having 
information typed, then copies made in some fashion, and finally 
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envelopes addressed and sent through normal mail channels. With their 
experience on the ARPANET, the subjects had been able to prepare 
information and electronically send it within minutes to multiple 
addressees simultaneously, anywhere on the network, thousands of miles 
apart. 

The other questions dealt with more specific areas of interest 
and are not of importance here. For example, one question asked "To 
what extent are you familiar with artificial intelligence applications?" 
and obviously most showed an increased familiarity since they were 
introduced to some artificial intelligence applications during the 
experiment. One similar type of detailed question was revealing from a 
man-machine interface viewpoint. That question asked "To what extent 
does the ability to type enhance your ability to communicate?". After 
the experiment, the subjects showed a significant, positive increase in 
their responses, indicating that they now felt the ability to type was 
even more important. This has significant implications for the future 
if the current trend continues toward the use of more computer systems 
in our daily lives. Current trends are toward more interactive use via 
computer terminal, and if other modes of data entry are not developed, 
the acquistition of typing skills in ones's earlier schooling years will 
be extremely valuable in later years. 

Typically, after students in a computer type class have finished 
their course, their use of computers and computer systems will drop to 
practically zero, if in the following weeks and months, they are not 
taking more computer courses or working on thesis research. A further 
verification that the subjects in this experiment had been motivated or 
"turned on" by the experimental experience they received, is indicated 
in accounting data for the following two months after the experiment had 
ended. Although these subjects were taking no courses requiring use of 
a computer and none of them were working on thesis research, they have 
averaged 8 hours per week per subject using the facilities and research 
products provided them during the experiment and this has continued for 
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two months up to the time of this writing. 



Another interesting aspect revealed in the subjects' responses 
was a significant change in their attitude toward deleting files of 
information. After the experiment, they felt much more comfortable in 
creating and destroying files of information. This is probably due to 
their previous experiences in which most computer system users are told 
very specifically how to delete information, because it will be gone 
forever if one deletes it. The ARPANET has been designed however, such 
that a user may delete information, but if the user realizes at anytime 
before he logs out, that he made a mistake and really needs the file of 
information, he can simply undelete it and he will have it restored in 
its original form. This is a nice feature for the user and certainly 
had an influence on the subjects during this experiment. 

A purely subjective observation by the authors also revealed 
that subject interest levels started out very high and remained so 
throughout the experiment. This observation is based not only on 
accounting data, but on the enthusiasm and excitement exhibited by the 
students in daily contacts with them. 

At the end of the experiment, the subjects were also asked to 
indicate which of the query systems they preferred ... the natural 
language query system or the less natural, tree structured, query system 
described earlier. Their preference was to be based on the man-machine 
interaction with the two systems and on how they could formulate 
queries. Their preference was not to be based on what type of 
information they could obtain from the system. The subjects were 
divided into two groups called OPERATORS and SUPPORT. OPERATORS means 
airplane pilots, ship drivers, etc. SUPPORT refers to individuals in 
communications, intelligence, etc., whose main function is in support of 
the operational areas. This is analagous to the staff function in any 
large organization. Six OPERATORS chose the natural language system and 
two OPERATORS chose the tree structured query system. However, seven of 
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the SUPPORT group chose the tree structured query system and two of the 
SUPPORT group chose the natural language system. Arranging this data in 
a 2 X 2 matrix, the Fisher Exact Probability Test (Siegel, 1956) was 
applied and showed a significant difference (p < .05) in the two groups 1 
preferences. The OPERATORS definitely preferred the natural language 
type query system, and in reality, they would usually be flying planes, 
etc., i.e. performing the "command” functions, and only occasionally 
have need to go into a command center and enter their own query into a 
system. As such they would prefer to not have to remember details of a 
system but rather enter their own "natural" query. The SUPPORT group 
definitely preferred the tree structured query system which allowed them 
to use a "shorthand" version of query entry, and thus enter queries much 
faster. In reality, the SUPPORT group would use such systems on a dailv 
basis. This revealing aspect of query system preference indicates that 
although the current trend today is toward development of natural 
language systems in many field3, one should carefully examine the daily 
user to determine if the additional cost of a natural language system is 
really worth the amount of use the system will receive, or would a less 
natural language system in fact serve the users 1 purposes better? 

CONCLUSIONS 

The transfer of technology from designer to user is often a 

difficult process and many times the transfer does not occur because the 

technology designer did not consider how the technology would be used by 

the end user. In this experiment, a broad range of research technology 

products were presented to users. However, the research technology 

products were all ones that had considered human factors in their 

initial design stages, and as a result, subjects found the products easy 

to use. Consequently, user perceptions of computer systems and computer 

software projects were greatly enhanced, and in addition, user 
and 

motivations^ interest levels in working with these systems were greatly 
elevated and have remained so. 
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Two comments might be made in closing. Since the subjects in 
this experiment were professional military officers, with current 
operational experience, they provided timely, real-world comments to 
designers on how to improve their products, with such improvements being 
made immediately in several cases. Finally, a side benefit derived from 
providing each subject with his own portable terminal at home, was that 
the subject could work at home at night and still be with his family, as 
opposed to the more typical situation in which the student SDends many 
nights at the computer center and the family begins to wonder if he 
doesn’t really live at the computer center. 

An overview of this experiment strongly suggests that a viable 
method of technology transfer may be through special programs which 
"invite" the subject or user to become familiar with the technology and 
evaluate it. It might also be suggested that the human interface with 
automated systems, expecially decision support systems, can be 
substantially enhanced through a properly designed exposure to the 
technology and an opportunity to use it without fear of failure. 



NOTE; Follow on observations of the 2nd 
indicated the same trends and results as 



3 

and 3rd C classes also 
found in this study. 
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